Enable ipex and other optimizations #1628
Conversation
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
I am cleaning the CPU and XPU tests, process 50% |
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
| quant_state.blocksize, | ||
| quant_state.shape, | ||
| quant_state.dtype, | ||
| ) |
There was a problem hiding this comment.
Is there reason why this change can't be in bitsandbytes/backends/cpu/ops.py?
There was a problem hiding this comment.
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
|
pytest --ignore test_optim.py --ignore test_triton.py --ignore test_cuda_setup_evaluator.py CPU previous: 378 passed, 1537 failed, 1638 skipped, 197 xfailed, 153 warnings in 613.27s XPU previous: not enabled It also could pass all transformers tests I also updated the installation guide. Hi @matthewdouglas . Please take the next round review. |
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
docs/source/installation.mdx
Outdated
| @@ -316,15 +316,29 @@ pip install -e . # `-e` for "editable" install, when developing BNB (otherwise | |||
| > [!TIP] | |||
| > Intel CPU/XPU backend only supports building from source; for now, please follow the instructions below. | |||
|
|
|||
| It does not need compile CPP codes, all required ops are in [intel_extension_for_pytorch](https://pytorch-extension.intel.com/), please follow the instruction to install ipex. | |||
| It requires [intel_extension_for_pytorch](https://pytorch-extension.intel.com/), please follow the instruction to install ipex. | |||
There was a problem hiding this comment.
I would expect IPEX to be optional. Especially so for CPU on Windows or for Linux/macOS on aarch64.
There was a problem hiding this comment.
Yes, I have updated the limitation.
Signed-off-by: jiqing-feng <[email protected]>
|
Hi @matthewdouglas . Please check if there are anything missed before merging. We can discuss the tests in this issue #1637 |
Signed-off-by: jiqing-feng <[email protected]>
bitsandbytes/cextension.py
Outdated
| if not ipex_cpu: | ||
| logger.warning( | ||
| "The installed version of bitsandbytes was compiled without IPEX support. " | ||
| "You can install ipex by running `pip install intel_extension_for_pytorch`to get better performance if you use the Intel CPU.", | ||
| ) |
There was a problem hiding this comment.
This seems like extra noise that we'd want to avoid.
Something to point out is that we still plan to ship libbitsandbytes_cpu in our wheels, so for most users, it's going to load a CPU, CUDA, or eventually ROCm or Metal library and we'll hit this logging line. At most we should really only raise this warning when:
- We're on a platform with IPEX CPU support. My understanding is this is limited to Linux x86-64.
- We expect the user to be using CPU, i.e. no CUDA, XPU, or MPS accelerators available.
On torch >= 2.6 we could just usetorch.accelerator.is_available()and on older versions I think we can overlook privateuse1 backends like HPU or Ascend NPU. - There's some expectation of IPEX being beneficial. We don't want to prompt users to install it if e.g. it needs AVX512 or AMX support to be effective. This is something I can't speak to directly but defer to Intel folks to determine.
Any other thoughts @Titus-von-Koeller ?
There was a problem hiding this comment.
You were right. I also agree that the log should only exist if no devices like cuda/xpu are available and the CPU is an Intel product.
There was a problem hiding this comment.
I have changed it, please review again. Thanks!
Signed-off-by: jiqing-feng <[email protected]>
Signed-off-by: jiqing-feng <[email protected]>
|
Hi @matthewdouglas . You can see some CPU ops are implemented by pure python, which works for all devices. Could we move these pure python implementations to the default folder? As some of these ops could be reused by XPU. |
Signed-off-by: jiqing-feng <[email protected]>
This PR enables ipex and other optimizations including:
Also, it fixed the parameter patch for cpu.
It could pass all transformers tests
After this PR merged, I will update the installation guide.
@matthewdouglas @Titus-von-Koeller